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ABSTRACT 



This paper statistically analyzes 15 years of NAVAIRSYSCOMHQ 
fixed-price incentive contract experience in the aircraft and missile 
procurement field. The relation of basic contract parameters to con- 
tract outcome is explored through regression and analysis of variance 
techniques . 

The inferences arising from the statistical analysis are combined 
with other information to draw conclusions regarding incentive contract- 
ing. The most important of these is that there is no evidence that the 
negotiated sharing ratio has any influence on the contractor during the 
performance of the contract. 
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I. INTRODUCTION 



A. FIXED -PRICE INCENTIVE CONTRACTS - THEORY AND PRACTICE 
1 . Theory (from the government's viewpoint) 

The basic parameters of a fixed-price incentive (FPI) contract 
are: target cost, range of incentive effectiveness (RIE), sharing 

ratio, ceiling price, and target profit. 

Fixed-price incentive contracts are intended to be used for 
engineering development, management support, and production contracts 
in which the uncertainty of cost is too large to allow a firm fixed- 
price (FFP) contract. In terms of cost, this means approximately + 

15> to 20 % variation about the negotiated target cost. If the range 
of cost uncertainty is larger than this, a cost reimbursable contract 
is indicated. If significantly smaller, a FFP contract should be 
considered . 

FPI contracts are further subdivided into those involving a 
cost incentive alone, and those having schedule and/or performance 
incentives as well as a cost incentive . Both the cost incentive and 
multiple incentive contracts may have firm target(s), FPIF, or suc- 
cessive targets, FPIS. The contracts analyzed in this thesis are FPIF 
and FPIS contracts with cost incentives only. 

Theory has it that the contractor will be influenced to effec- 
tively control costs and to make cost-associated trade-off decisions 
in a way favorable to his profit and therefore favorable to the govern- 
ment. The government's desires with regard to the cost outcome of a 
contract are communicated to the contractor in the form of the mutually 
agreed upon sharing ratio. This ratio, e.g., 80/20, says to the 
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contractor that his profit will increase $.20 for every dollar he saves 
in cost throughout the RIE, and vice versa. The RIE is chosen so as to 
encompass all likely contract cost outcomes including the most likely 
outcome (the target cost). Insurance against a massive cost overrun is 
provided by another parameter, ceiling price. This figure tells the 
contractor the maximum the government will pay, cost + profit, for com- 
pletion of the contract no matter what the cost outcome. It should be 
noted that for a cost outcome greater than the ceiling price, the con- 
tractor's profit is negative, i.e., a loss. Also for a range of cost 
outcomes less than the ceiling price, each dollar of extra cost comes 
out of the contractor's profit directly. In effect, the sharing ratio 
in this range is 0/100. This range is bounded by the point of total 
assumption (PTA) on the lower cost side, and unbounded on the higher 
cost side. 

The target profit is established upon, at least in part, the 
degree of financial risk accepted by the contractor as determined by 
the type of contract and where applicable, the sharing ratio. That : , 
an FPI contract with a $ 0/50 sharing ratio is assumed to be more fins - 
cially risky for the contractor than one containing an 80/20 sharing 
ratio. Further, a FFP contract is considered more financially risky 
than a cost plus incentive fee (CPIF) contract. 

The underlying assumptions of incentive contracting are: 

a. That the contractor seeks to maximize profit on each 

contract . 

b. That the most likely, optimistic, and pessimistic cost 
outcomes are established free of bias and with some degree of accuracy. 

c . That the contractor can control the cost of performance 
significantly during performance. 
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2. Practice 



In practice there is reason to believe that each of the previous 
assumptions is subject to considerable doubt. 

a. Profit maximization - Contractors appear willing to 
sacrifice profit on a particular contract to other goals such as long- 
term profit or simply survival. 

b. The most likely cost outcome, target cost, is subject 

to a number of types of bias. For example, a contractor may be willing 
to accept an unrealistically low target cost if the industry is charac- 
terized by overcapacity and high competition. Similarly, the government 
may accept an excessively optimistic view of future cost outcome if the 
competition for available funds is high within the government. The 
target cost negotiated in a sole-source situation may be unrealistically 
high if the government has no good basis of comparison. In which case 
the contractor can increase his profit by "accepting” an FFP contract 
and gain a 0/100 share of the underrun and a higher target profit to 
boot /~ref . 1 And lastly, it is not currently possible to predict 

future costs with any reasonable degree of accuracy on a technically 
complex item which has not been produced before. Further, contract 
negotiation is an adversary proceeding in which each side does its best 
to protect its own interests. Such a proceeding cannot be characterized 
as a cooperative search for truth in the establishment of contract tar- 
get cost. For example, the contractor may raise his negotiation objec- 
tive with regard to target cost to compensate for the risk of loss 
imposed by the existence and value of the sharing ratio. The selection 
of three costs, pessimistic, most likely and optimistic, does not de- 
crease the uncertainty in the cost estimate unless valid information 

is available upon which to base the cost probability distribution 
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inherent in the three costs. In the author's opinion, such information 
is generally not available and the seeming decrease in cost uncertainty 
is illusory. 

c. Contractor control of cost during performance - 
In order to control costs during performance, the contractor must be 
able to predict cost outcomes sufficiently in advance and take effec- 
tive action. Attempts by a contractor to make significant improvements 
in efficiency during performance of a contract may result in managerial 
disutilities and may increase rather than decrease costs /""ref. 2_7 • 

B. OBJECTIVE OF THE ANALYSIS 

The objective of the analysis presented subsequently was to draw 
statistical inferences as to the readily quantifiable contract para- 
meters which have a statistically significant effect on contract out- 
come in percent deviation from target cost. The chosen parameters are 
ones that are measurable and known at the time the contract is signed 
or shortly thereafter. These parameters are target fee, sharing ratio, 
ceiling price, target cost, scheduled length of performance, number of 
articles procured, the number of aircraft and missile contracts initiated 
that year, and the year performance started. As originally scheduled 
contract performance was not available, actual length of performance 
was substituted therefor. 

A further objective of this analysis was to generate, if possible, 
a model which explained a significant amount of the variation from 
contract to contract in the outcome percent deviation from target cost 
using the contract parameters listed below. The utility of such a 
model is obvious in the predictive sense. 
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Percent deviation from target cost was determined to be the depen- 
dent variable for most of this analysis because of the three measurements 
of contract outcome * cost* schedule * and performance * cost is the one 
which is most readily measured and publicized. Further* the adverse 
effects of system performance degradation from initial estimates and 
schedule slides are difficult to identify and measure. 

The following factors in contract outcome may have been important 
but were unavailable: 

1. Initial contractor estimates of contract cost. 

2. Initial governmental estimates of what the contract should 

cost . 

3. The degree of technical risk involved in the article 
* purchased. 

k* The changes in contract cost generated by schedule changes* 
engineering change orders* governmental failures to deliver government 
furnished data* mterial and equipment on time. 

5>. Renegotiation of fees. 

These unmeasured variables and others can be expected to result in 
a rather large random. variation in contract outcome as regards devia- 
tion from target cost. If this variation is indeed random rather than 
systematic and is distributed approximately normally* useful inferences 
can be drawn from regressions and etc.* that do not explain a large 
part of the variation. 

C . HYPOTHESES 

In the opinion of the author* the following hypotheses could reason- 
ably be made regarding the effects of the measured contract parameters 
listed in B. on percent deviation from target cost: 
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1 . Length of performance of the contract - it might reasonably 
be expected that cost deviation would increase (that is positively or 
in the direction of cost overrun) as the length of performance increases 
This might be true because of the difficulty of projecting costs into 
the far future , the general inflationary trend experienced, and the 
increased opportunity afforded to make changes.. 

2. Target profit - Expectation would be that larger target 
profits would be associated with larger positive deviations from target 
cost because theoretically target profits are established on the basis 
of the risk involved in the contract and other factors. 

3. Sharing ratio - Actual costs moved in the negative direction 
(toward cost underrun). The theory of cost incentive contracting is 
that the stimulus provided by an increased sharing ratio will cause the 
contractor to at least attempt to operate more efficiently. Of course, 
it may also stimulate him to negotiate for a higher target profit and/or 
a higher target cost. 

li. Ceiling price - A higher ceiling price would be expected 0 
result in less attention to incurred cost and thereby tend to influen 3 
deviation from target. cost in a positive direction. All other things 
being equal, a higher ceiling price (in terms of percent of target 
cost) moves the point of total assumption further from the target cost. 
If the actual cost of performance exceeds the point of total assumption, 
each dollar of additional cost comes directly from the contractor's 
profit on the contract. In effect the contract becomes a firm fixed- 
price contract (100$ sharing ratio) beyond the point of total assumption 

3. Target cost - The author does not care to hypothesize on 
this variable's effect on deviation from target cost. As the dependent 
variable in this analysis is the ratio of cost deviation to target cost 
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expressed in percent, all other things being equal, increases in target 
cost would result in a decrease in the dependent variable. On the 
other hand, to the degree that an increased target cost may reflect 
increased complexity, increased time of performance, etc., an increase 
in target cost might result in a positive change in deviation from 
target cost. Notwithstanding the clouding effect that choosing such 
a ratio for the dependent variable has on this independent variable, 
target cost, it was considered necessary to permit meaningful compari- 
son among contracts which vary in target cost from under $1 million to 
over $200 million. 

6. Number of articles procured - Deviation from target cost 
was expected to vary negatively as the number of articles procured, 
i.e., the larger the number of articles procured the less positive or 
more negative cost deviation was expected to be. The larger the number 
of articles procured under a particular contract the less the develop- 
mental aspects of a basically production contract should effect actual 
total cost. Further, larger production quantities would be expected 

to result in operation on more favorable portions of the learning curve 
for more of the contract performance. 

7. Number of Navy aircraft and missile contracts signed the 
same year as a particular contract - It was expected that the larger 
the number of contracts signed, the less positive or more negative de- 
viation from target cost would be. The diversion of facilities and 
personnel from defense to non-defense work is at best a painful and 
expensive alternative for major aerospace contractors. To a lesser 
degree the diversion from aircraft and missile production to other 
defense production is also painful and expensive. In some cases, e.g., 
Grumman, the diversion from Navy aircraft and missile work to similar 
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work for the Air Force or Army is not particularly attractive. These 
things are considered true irrespective of the size of the contractor's 
investment in aircraft and missile production facilities because of 
the psychological barrier to entering a field where the rules of engage- 
ment are largely unknown from a field in which one is expert. Further, 
a Rand Corporation report indicates that at the same time that the num- 
bers of Navy aircraft and missile contracts were decreasing a like 
phenomena obtained in Air Force contracts / ref. 3 _7 at least as regards 
total contract dollar value. In sum then, exit from the Navy aircraft 
and missile market and entry into another was not easy so the effect of 
a decreasing number of available contracts should be a tendency toward 
a more optimistic view on the contractor's part of estimated costs 
particularly in competitively negotiated procurements . If such were the 
case, the final result could be expected to be a positive change in de- 
viation from target cost. 

8. The year the contract was signed - The author's intuitive 
expectation was that contracts signed in the later years of the period 
covered by the data would show more positive deviation from target cost. 
The steadily increasing publicity concerning cost overruns during the 
period would indicate that that was the case. The other variables' 
independency of the year the contract was signed is certainly open to 
question. Any of the other independent variables may prove to be time 
dependent which will reduce this variable's usefulness as a predictor. 
Other than as a reflection of changing contractor and/or government 
policy, the author is hard pressed to explain a systematic variation 
of deviation from target cost with time if such, in fact, exists. 
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II. DESCRIPTION OF DATA 



A . SOURCE 

The raw data were obtained by the author through the good offices 
of Mr. M. E. Biciocchi, Naval Material Command, code 022B. 



B. NATURE OF THE DATA 
1 . Form 

The data encompass all Naval aircraft and missile fixed-price 
incentive contracts completed, redetermined and approved by ONM during 
the period 195k through 1966. The information listed in the data for 
each contract is: 



a. Contract period 

b. Contract number 

c. Contractor 

d. Nomenclature and number of items procured 

e . Negotiated target cost 

f. Final cost 

g. Percent deviation from target cost 

h. Incentive profit pattern in terms of: 

(1 ) Target profit 

(2) Sharing ratio 

(3) Ceiling price in percent of target cost 

i. Final profit rate 

j . Target profit 

k. Final profit 



2 . Quantity of Data 



The data contained 1 78 individual data points (contracts) from 
which field one was deleted because it dealt with the procurement of 
spare parts. Thus the population analyzed contained 177 contracts. 

3 . Period 

Although the earliest contract in the data was signed in 1 9h9 
and the latest in 1963, it was considered that results useful in the 
future could be obtained therefrom. It would appear that future Navy 
procurement will more closely resemble that of the 1 9h9 through 1 963 
time frame than the later McNamara era. It is the author's opinion that 

'i 

data taken from Mr. McNamara's total package procurement contracts would 
be of little value in attempting to draw inferences useful in the future 
when total package procurement is so vigorously eschewed. 



C. DATA SUMMARY 



1 . Variable Means and Standard 


Deviations 




Variable 


Mean 


Standard 

Deviation 


Contract period (months) 


60.9 


19.3 


Target profit (percent) 


9.0 


0.8 


Sharing ratio (percent) 


19.3 


8.1 


Ceiling price (percent) 


123.2 


5.1 


Target cost (millions of dollars ) 


U6.9 


U 6.6 


Number of articles procured 


377 


1 2 ) 4.8 


Number of contracts signed per year 


17 


8 


Deviation from target cost (percent) 


- 1 .3 


9-1 



2. Comment on Variables 

Although target profit is intended to vary with sharing ratio 
and other factors, in fact, approximately 68 $ of negotiated target 
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profits fell within Hess than one percent of the mean, 9$. This is 
surprising considering that the data covers a period of 1 5 years and a 
multitude of different procurements. It is tempting to suspect that 
target profit was close to a "magic number" with a mode of, say, 10$. 

Ceiling price seems to fall rather neatly into the same category 
since the standard deviation is only five percent, and the mean, 123 . 2 $, 
falls close to the mode, 125$. 

3 . Miscellaneous 

The data includes purchases of combat and utility piloted air- 
planes, helicopters, blimps, drones, missiles, and airborne electronic 
equipment, e.g., radars. Twenty-three contractors are represented in 
the data. Of these, thirteen completed five or more contracts during 
the period. Six completed 10 or more contracts: 

Ling-Temco-Vought (10 contracts) 

Convair (10 contracts) 

North American (13 contracts) 

Douglas (23 contracts) 

Grumman (21; contracts ) 

Lockheed (30 contracts) 

On the' contracts contained in the data, the Navy spent a total 
of $ 8,885,589,000 of which $ 8 , 1 3 hj 3 i| 0,000 were actual costs and 
$ 751 ) 2^9)000 actual profits. The average profit rate realized by the 
contractors was 9 . 2 % of actual costs. During the period of the data, 
defense contracting, at least in this area, was an extremely profitable 
business even on the basis of sales. Return on investment figures 
would, of course, be even more attractive given the large amount of 
government owned facilities involved. 
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III. DATA ANALYSIS METHODS 



A . UNEAR REGRESSION 
1 . Theory 

Linear regression is a technique for making statements as to 
the degree of linear association between a criterion (dependent) and 
one or more predictor (independent) variables using the method of least 
squares. If evidence exists that suggests that the form of one or more 
independent variables should be of a higher degree, the original vari- 
ables can be trans gene rated to fit the linear assumption. The following 
assumptions are required in order to make probability statements about 
single or multiple linear regression: 

a. That the relation between the expected value of the 

dependent variable ( Y ) and an independent variable (X) (or J? or *3 

or X^ *Xg or X^ /^ 2 ) etc., as previously mentioned) is linear. 

b. That the error distribution is constant or homosceda: ic. 

c. That no variable which is apt to effect the dependent 

variable is excluded from the equation. This assumption is not inpo: * 

tant if the objective is forecasting. 

d. That the differences between actual and estimated values 
of the dependent variable are normally distributed. 

Though these assumptions are not always justified in this analysis, the 
insight to be gained from the linear regression is considered more im- 
portant than precision. 

2 . Computer Code 

The computer code used by the author for single and multiple 
linear regressions on the dependent variable was a part of the 
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International Business Machines Scientific Subroutine Package labelled 
REGRE. REGRE provides the following output for a particular regression 

a. A list of the dependent and independent variables included 
in the regression along with their means and standard deviations. 

b. A simple correlation coefficient (r) for each independent 
variable which measures the degree to which variation in the dependent 
variable is linearly associated with variation in that independent 
variable. The simple correlation coefficient has a range from -1 to +1 
Larger absolute values of the simple correlation coefficient indicate 
greater correlation while the sign indicates dependent variable varia- 
tion in the same direction as the independent variable (+) or opposite 
to the independent variable ( - ) . 

c. The regression coefficient determined for each independent 
variable included in the regression (b^ * **,b ). 

d. The standard error for each regression coefficient. 

e. A measure of assurance that each regression coefficient is 
not statistically different from zero by use of the Student t statist c 

f. The value of the estimated dependent variable when the in- 
dependent variables all equal zero (intercept). 

g. The multiple correlation coefficient (R) from which the 

2 2 

coefficient of multiple determination (R ) may be obtained. R ranges 
between 0 and 1 and represents the proportion of variation in the de- 
pendent variable that is accounted for by the net linear association 
of all of the independent variables included in the regression. 

h. The standard error of estimate for the regression. 

i. The degrees of freedom^ sums of squares and mean squares 
associated with an analysis of variance for the regression and the 
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resulting Snedecor F statistic which infers the degree of assurance one 
may place in the fact that there is a systematic linear variation of 
the dependent variable with the included independent variables. 

3 . Rationale for Linear Regression 

Although there is some support in the literature for a non- 
linear association of sharing ratio with deviation from target cost, 
the analysis is based on theories of risk aversion or easy exit from 
the defense market /~ref. U_7 which the author believes to be unrealis- 
tic. The author's opinion on the ease of exit from or entry into the 
defense market has already been stated in section I.C.7. In the 
author's opinion contractors are willing to accept a high risk of de- 
creased or no profits to buy into a contract because follow-on contracts 
can be expected to be profitable. Contractors may also expect to im- 
prove their profit position through change orders (constructive or 
otherwise) during performance of the contract. A toughening government 
position on this sort of activity is a creature of the late 1960's and 
1 970's . 

In short then, the author has chosen to confine himself , except 
in one case, to linear combinations of first-degree variables, 
li. Stepwise Linear Regression 

The form of multiple linear regression actually used by the 
author in the final analysis was stepwise regression. In stepwise 
regression, a sequence of multiple linear regressions are computed in 
a stepwise manner. At each step one independent variable is added to 
the regression equation on the basis of which of the remaining (unused) 
independent variables makes the greatest reduction in the error sum of 
squares. Equivalently, the variable added next is the variable which 
has the highest partial correlation with the dependent variable partialed 
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on the variables which have already been added; and equivalently it is 
the variable which, if it were added, would have the highest F value. 

The computer code used for stepwise regression was BMD02R 
developed by the Health Sciences Computing Facility at the University 
of California at Los Angeles. BMD02R also produces a correlation 
matrix of each independent variable with all of the other independent 
variables considered for the regression equation. It also outputs a 
plot of residuals versus computed values of the dependent variable for 
examination for systematic trends, if any. The intercepts, Multiple R, 
standard error of estimate, and analysis of variance are produced after 
each step as in the linear regression code discussed previously, REGRE. 

5>. Linear Regressions on Segments of the Data 

Linear regressions were performed for each contractor having 
five or more contracts during the period relating the year the contract 
was signed and the number of contracts signed that year to predict values 
of percent deviation from target cost. 

The plot of percent deviation from target cost versus the year 
of the contract in the case of Lockheed revealed a sine wave pattern. 

A special linear regression was done on the Lockheed data using the 
cosine of a function of the year signed as the independent variable. 

6. Target Profit as a Dependent Variable 

To investigate the degree to which target profit depends upon 
the negotiated sharing ratio and length of contract, linear regressions 
were performed using target profit as the dependent variable and sharing 
ratio and length of contract respectively as independent variables. 

7. Regression Independent Variables 

The following variables available in the data were considered 
to be suitable for linear regression: 
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Symbol 



Description 



X 

X 



1 

2 




*11 



Length of contract period in months 
Actual cost deviation from target cost 
in percent of target cost (dependent 
variable ) 

Target profit in percent of target cost 
Sharing ratio in terms of contractor's 
share of the cost risk in percent 
Ceiling price in percent of target cost 
Target cost in millions of dollars 
The number of articles procured 
The number of Navy aircraft and missile 
contracts signed the same year as a 
particular contract 

The year the contract was signed , coded 
by subtracting 1 9^8 

The ratio of the target cost and the n iber 
of articles procured in millions of do lars/ 
article 

The product of the target cost and the 
number of articles procured in article - 
millions of dollars 



B. ANALYSIS OF VARIANCE 
1 • Theory 

Analysis of variance is a way of dividing total variation in 
experimental data into components that can be assigned to specific 
sources. The objective is to test the statistical significance of 



21 



differences among average responses caused by some variable after making 
allowances for other variables. The label* analysis of variance, is 
appropriate because if the mean responses of the test objects are 
different among groups, then the variance among groups will exceed the 
independently computed within-group variance. 

Analysis of variance has some rather severe drawbacks when ap- 
plied to nonexperimental or observational data. First, when some 
classifications of the data contain one or less data point, a within- 
group variance can not be computed and the mathematical rigor of the 
analysis is severely reduced. This problem can be ameliorated by 
pooling of some treatments such that several data points fall in each 
classification. Second, the basic analysis does not provide informa- 
tion as to which treatments ) were statistically different. The F test 
performed only indicates that one or some of the populations were 
actually different. Further investigation as to which treatment or 
treatments were actually different from the others can be conducted 
using the Scheffe comparison test which will be discussed later. Thi i, 
gradual changes in the means in the data through the treatments will 3 
discovered in the analysis of variance but not be isola table by the e .b- 
sequent comparison test. In this case, the effect of the particular 
variable may be investigated through regression or a combination of 
regression and analysis of variance, analysis of covariance. 

The basic assumptions of the parametric analysis pf variance 

are : 

a. That the observations are independently drawn from normally 
distributed populations. This assumption makes the analysis parametric. 

b. That the populations all have the same variance. 
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c . That the means in the normally distributed populations are 
linear combinations of "effects" due to rows and columns, i.e., that 
the effects on the dependent variable of the factors included are 
additive . 

2 . Computer Code 

The computer code used by the author for one-way analysis of 
variance was BMD01V developed by the Health Sciences Computing Facility 
at the University of California at Los Angeles. BMD01V outputs: 

a. Treatment means and standard deviations 

b. Within -groups, between-groups, and total sums of squares. 

c. Within-groups, between-groups, and total degrees of freedom. 

d. Within-groups and between-groups mean squares. 

e. F ratio for the test of the hypothesis Hq 5 mean-) = mea^ 
=****= mean, . 

K 

More than one-way analyses of variance were not used by the 
author because of the prohibitive severity of the empty cell or single 
observation cell problem which arises in multi-way analyses. For ex 
ample, a three-way AN OVA on contractor (there are 23 of them), year 
05 of them) and item (7 of them) would contain 23 X 15 X 7 = 2l|l5 
cells. Since there are 177 data points, at the least, 2i|1 5 - 1 77 = 

2238 would be empty, and the majority of the filled cells would con- 
tain only one data point. 

3. Rationale for Use 

The author's rationale for employing a parametric analysis of 
variance is essentially that stated under linear regression. 

U. Variables 

The factors (variables) chosen for one-way analysis of variance 

were: 



23 



a. Factor one - contractor with the individual contractors 



being the levels 


of that factor (contractors that had more than one 


contract during 


the period). 


b. Factor two - type of equipment procured using the following 

levels : 


0) 


Utility airplanes 


(2) 


Combat airplanes 


(3) 


Missiles 


Ot) 


Blimps 


(5) 


Helicopters 


(6) 


Pilotless airplanes (drones) 


(7) 


Airborne avionic equipment 



C. SCHEFFE COMPARISON TEST 

1 . Theory 

If an analysis of variance has resulted in the rejection of the 
equal mean hypothesis, the Scheffe comparison test (S-test) may be used 
to compare levels two at a time (or more) to identify which levels were 
significantly different from others, if any. Again, a gradual change 
over the levels will not be identified. 

2 . Computer Code 

Because a preprogrammed routine for the S-test was not found 
available by the author, a computer code was written by the author. 

That code is contained in Appendix A. No attempt was made to write a 
generally applicable code. The program contained in Appendix A is 
tailored to fit the data being examined in this thesis and to use the 
level means, etc., available from the preceeding analysis of variance. 
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3 • Application 



Analyses of variance and Scheffe comparisons were conducted 
upon the entire data and some individual contractor data. Individual 
contractors so treated were those that completed enough contracts 
during the period to make such an analysis feasible. Specifically, 
the Lockheed, Grumman and Douglas data, classified by the year the 
contract was signed, and pooled as necessary to have at least two 
observations per group, were so analyzed. 
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IV, RESULTS OF DATA ANALYSIS 



A. LINEAR REGRESSIONS 
1 . All Contractors 

a. Stepwise Linear Regression 

The stepwise linear regression discussed in paragraph 

II.A.lj. resulted in the following multiple linear regression equation: 

Percent deviation from target cost = -U2.1* + 0.1 2 8 X^ - 1 .68X^ 

+ 0.397X - 0.297X o + 0.823X. - 0.0000 

5 8 9 il 

The independent variables entered the equation in the order 
X Q> Xy, X^ , , Xy, X^. When the stepwise regression routine termi- 

nated the computation, the lowest F-ratio for a variable included in 
the equation was 7.53U5* The highest F-ratio for a variable not in- 
cluded was 1 .1 953 * When the routine was forced to start with each of 
the variables other than Xg, the resulting equation was the same unless 
the routine was forced to include a variable which would not have been 
otherwise included because of low F-ratio. 

The linear regression variables are included here again to 
ease interpretation of the results: 



Symbol 

X 1 



Description 

Length of contract period in months 
Realized deviation from target cost in 
percent of target cost (dependent variable) 
Target profit in percent of target cost 
Sharing ratio in terms of the contractor's 
share of the overrun or underrun 
Ceiling price in percent of target cost 
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Symbol Description 

X 6 Target cost in millions of dollars 

X ? The number of articles procured 

The number of Navy aircraft and missile 
contracts signed the same year as a 
particular contract 

The year the contract was signed, coded 
by subtracting 1 9h 8 

The ratio of the target cost and the number 
of articles procured in millions of dollars/ 
article 

The product of the target cost and the 
number of articles procured in article- 
millions of dollars 

The F-ratio for the whole regression is 9*559 which indi- 
cates that the level of the test is higher than 0 . 005 * That is, the 
probability that all of the regression coefficients are not zero is at 
least 0.995* 
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The F-ratio test for each regression coefficient in the 
equation above along with the associated probabilities are listed 



below: 








Regression 

Coefficient 


Value 


F-ratio 


Probability 
bj_ / 0 


b 


0.128 


12.70 


0.995 


b 3 


-1 .680 


U* 67 


0.950 


b 5 


0.397 


8.37 


0*995 


b 8 


-.297 


7*03 


0.990 


b 9 


0.823 


9*51 


0.995 


b 11 


-.OOOOli 


7*53 


0.990 
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were formed from two 



The additional variables, X^ and , 
of the original variables, X^ and X^,, which were not significant in the 
regression individually and were not closely correlated with each other 
as determined from the correlation matrix. The correlation of X^ with 
Xy was -.023. 



X 
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' x 6 /x 7 

■ V X 7 



The variables considered in the regression but not included 

were : 



Regression 

Coefficient 


F-ratio 


Probability 
b ± / 0 


b u 


0.80 


' less than 0.73 


b 6 


0.68 


" " 0.73 


b 7 


0.38 


" " 0.30 


b 10 


0.90 


» " 0.73 



The value of the multiple correlation coefficient (R) fo. 

this regression was 0.302. Therefore, the value of the coefficient 

2 

multiple determination (R ) for this regression was 0.232, i.e., 
approximately 23$ for the variation in the dependent variable (percent 
deviation from target cost) is accounted for by the net linear associa- 
tion of independent variables, X^ , X^, X^, Xg, X^ and X^ . 

The standard error of estimate for the regression is +_ 7.97 
percent deviation from target cost. That is, approximately 68$£ of the 
estimates from the regression equation fall within _+ 7-97 of the ob- 
served values of percent deviation from target cost whose mean was 

-1 .23. 
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b. Percent Deviation from Target Cost Versus Year Contract 

Signed 

A linear regression was performed relating the percent de- 
viation from target cost to the contract year alone to get a historical 
perspective . The resulting equation was : 

% deviation from target cost = — U • 9U + 0.37 3X^ 

The computed t-value for this regression was 3*28 or equivalently the 
F-ratio was 10.73, both meaning in this case that the probability that 
the value of the regression coefficient was actually zero was . 003 * 

The coefficient of correlation was ,2h0 and the proportion of variation 
in the dependent variable accounted for is approximately 0.06. The 
standard error of estimate was +_ 8 . 82 . 

c. Target Profit (X^) Versus Other Contract Parameters 

A series of single linear regressions were conducted to 
determine if any other contract parameter known to both sides during 
negotiations was a good predictor of the negotiated target profit. 

The results of these regressions were as follows: 



Independent 

Variable 


Intercept 


Hi 


R 




Standard 
Error of Est. 


Probability 
bi / 0 


X 1 


9.60 


-.01 02 


.232 


.03 


0.828 


0.993 


x u 


8.37 


.0320 


• 301; 


.09 


0.811 


0.993 


x ? 


8.92 


.00017 


.23U 


.06 


0.823 


0.993 


x 6 


9.07 - 


.001 82 


.100 


.01 


0.8U7 


0.730 


x ^ 




-.0329' 


U19 


.10 


0.807 


0.993 



A multiple linear regression was performed using dependent 
variables , X^, X^ and Xy. The results of this multiple linear 
regression were: 
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Independent 

Variable 


b i 


t-value 


Probability 
b ± / 0 


x i 


-.001*37 


1 .36 


0.900 


\ 


.0182 


2.30 


0.975 


x 5 


-.0395 


3.31 


0.995 


x ? 


.0001 


2.08 


0.975 


Intercept 




13.37 




R 




0.1*37 




R 2 




0.19 




Standard Error of Estimate 


0.773 




F-value 




8.06 




Probability that b^ 


/ 0 


0.995 





Variable X^ (Target Cost) was not included in the final 



regression equation because the t-value (.01 ]*) did not have a high 
enough probability that the regression coefficient (b^) was not equal 
to zero (less than 0.75) • 

2 . I ndividual Contractors 
a. Grumman 

The plot of percent deviation from target cost versus year 
of the contract for Grumman 1 s contracts contained in Figure 1 displayed 
an apparent annual trend. This apparent trend suggested a linear re- 
gression on percent deviation using the year the contract was signed 
(X^) as the independent variable. The results of this regression were: 



Intercept -12.31; 

Regression Coefficient (b^) 1 .57 

Correlation Coefficient (R) 0.631 

o 

Determination Coefficient (R ) 0.1*0 
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YEAR CONTRACT SIGNED 



Std. Error of Estimate 



+ 8.20 



F-value 

Probability that b^ / 0 0.99% 

A similar linear regression of percent deviation versus the 
number of contracts signed the same year as the observed contract (Xg) 
was performed. The results were: 



Intercept 9*68 

Regression Coefficient (bg) -0.716 

Correlation Coefficient (R) 0.667 

2 

Determination Coefficient (R ) O.kZ 

Std. Error of Estimate 7-88 

F-value 1 7 • 65 

Probability that bg / 0 0.995 



Because of the predictive power displayed by variables Xg 
and X^ in the case of the Grumman data, the author was moved to attempt 
to generate a Grumman model for percent deviation from target cost. 

This was done using the stepwise linear regression routine previously 
used for the same purpose with all of the data. 

One of the uncertainties of stepwise regression is that the 
resulting value of the multiple correlation coefficient (R) may vary 
depending upon the first independent variable used. This uncertainty 
becomes particularly important when the list of independent variables 
considered contains two rather powerful predictor variables (Xg and X^) 
which are closely correlated (correlation of Xg with X^ = -O. 838 ). In 
this case, the routine picks the most powerful variable and the other 
is more or less automatically eliminated from further consideration by 
its high correlation with the first variable used. To eliminate this 
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uncertainty of result, the stepwise regression was conducted with all 
independent variables free and then another stepwise regression was 
conducted in which the routine was forced to start with the previously 
unused variable. The results of these two stepwise linear regressions 
were : 



(1 ) All Independent Variables Free 



Regression 

Coefficient 


Value 


F-ratio 


Probability 
b t / 0 


b 8 


-0.688 


19.71+ 


0.995+ 


b io 


8.91+8 


3-72 


0.900+ 


Intercept 




1+.95 




Correlation Coefficient (R) 


0.752 




Determination 


2 

Coefficient (R ) 


0.57 




Std. Error of 


Estimate 


17.11+ 




F-ratio for the Regression 


1 3 • 61+6 




Probability all b^ / 0 


0.995 






(2) Routine Forced 


to Use 




Regression 

Coefficient 


Value 


F-ratio 


Probability 
bj_ f 0 


b 1 


0.101 


2.26 


0 . 750 + 


b 3 


-10.8 


12.86 


0.995+ 


bo 


2.71+ 


39.19 


0.995++ 



Intercept 


72.22 


Correlation Coefficient (R) 


0.82 


Determination Coefficient (R^) 


0.66 


Std. Error of Estimate 


+6.1+3 
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F-ratio for the Regression 



13-20 



Probability all b^ / 0 0.995++ 

b. Lockheed 

A plot of percent deviation from target cost versus the 
year the contract was signed for Lockheed (Figure 2) also revealed an 
interesting pattern. This pattern looked to the author like a sine 
wave. The independent variable for a linear regression was determined 
to be a function of the year the contract was signed (X = cos(360/12 
(Xp - 2) + 90)) where the period of the wave was approximated as 12 
years. The regression equation used was then % deviation = a + bX. 

The results of this regression and an ordinary linear regression for 
comparison were: 



X 


= cos (360/1 2 (X ? - 2) + 90) 


11 

X 

VO 


Intercept (a) 


-2.3 


-U.31 


Regression Coefficient (b) 


2.U 


0.272 


Correlation Coefficient (R) 


• 387 


0.222 


Determination Coefficient (R ) 


0.19 


.05 


F -value 


li.928 


1 .1x57 


Probability that b / 0 


0.950 


0.75C 



c . Othe r 

Plots of percent deviation versus year for the other con- 
tractors displayed no discemable significant pattern. 

Linear regressions of percent deviation versus number of 
contracts signed that year for the other contractors failed to reach 
the probability that b / 0 required for significant statistics (0.950). 
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B. ANALYSES OF VARIANCE AND SCHEFFE COMPARISONS 
1 . By Contractor 

a. Analysis of Variance 

Preparatory to conducting a one-way analysis of variance by 
contractor, the data field was reduced by removing the data for those 
contractors who had only one contract during the period. Each remaining 
contractor was designated as a treatment level of the factor, contrac- 
tor. The sample size , mean percent deviation and standard deviation 
for the contractors (treatments) analyzed were: 



Treatment 


Sample 

Size 


Mean 


Standard 

Deviation 


Beech 


1* 


-2.78 


12.21 


Ling-Temco-Vought 


10 


0.59 


9.38 


Convair 


10 


-.37 


8.99 


Douglas 


2l* 


-2.11 


9.25 


Goodyear 


6 


-U.70 


ii .75 


Grumman 


2b 


-1 .99 


10 . 31* 


Hiller 


8 


7 .61* 


8.18 


Kaman 


6 


11.95 


12.97 


Martin 


7 


- 6.30 


7.35 


McDonnell 


6 


li.95 


11 .82 


North American 


13 


-1 -75 


6.07 


Vertol 


3 


-U.77 


5*1*8 


Ryan 


6 


-2.33 


3.60 


Sikorsky 


9 


-6.82 


1*«60 


Bell 


2 


U.65 


8.81* 


Lockheed 


30 


-2.50 


1**53 
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Sample 




Standard 


Treatment 


Size 


Mean 


Deviation 


Philco 


2 


u. 25 


17-1+7 


Raytheon 


3 


-8.20 


13.17 



The F-ratio for this analysis of variance was 2.76 which 
allows rejection at a significance level of .005 of the hypothesis 
that the treatment samples all came from the same population. That is, 
one may be .995 confident that the treatment samples did not all come 
from the same population. However, the analysis of variance does not 
allow selection of any treatment as being significantly different from 
another. That is, one may not say that Philco with a mean percent de- 
viation from target cost of 1 U • 25 is significantly different from 
Raytheon with a mean percent deviation of -8.20. 
b. Scheffe Comparison 

To continue the investigation started in a. above, a 

i 

Scheffe Comparison test was conducted using the F-value corresponding 
to the level of significance designated by statisticians as statis- 
tically significant, .05* The result was that none of the treatments 
(contractors) were different from any of the others at that level. 

The level of the comparison was successively reduced until differences 
began to appear. At a significance of 0.25 there was still no statis- 
tical differences between contractors. At a significance level of 
0.50 differences began to emerge, but this is the statistical equivalent 
of a toss-up, i.e., these differences were no more likely to exist than 
to not exist. 
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2 . By Type of Item Purchased 



a. Analysis of Variance 

To determine whether percent deviation from target cost 
were significantly different among different types of items purchased , 
a one-way analysis of variance was conducted in which the type of item 
was the factor and the treatment levels were the seven different types 
of items purchased which have been previously identified. The mean 
percent deviations, sample sizes, and standard deviations obtained were 





Sample 




Standard 


Treatment 


Size 


Mean 


Deviation 


Utility Airplanes 


21 


-k'kh 


1.22 


Combat Airplanes 


88 


-O .98 


8.52 


Missiles 


19 


-2. Ill 


10.82 


Blimps 


6 


-it. 70 


U. 75 


Helicopters 


28 


2.37 


11.08 


Drones 


8 


1 .08 


7 *1i7 


Airborne Avionics 


8 


-1 .18 


9.50 


The F-ratio for 


this analysis was 


1 .Ii5 which corresponds ti 


a level of between .25 and . 


.10. This level is 


insufficient 


to allow 


rejection of the hypothesis 


that the treatment 


samples came 


from the 



same population, therefore further investigation using the Scheffe 
comparison test was not justified. 

3. Individual Contractors by Year the Contract Was Signed 

Three of the contractors were selected for separate analyses 
of variance on the basis that they had more than 20 contracts during 
the period. The year the contract was signed was selected as the 
factor for one-way analyses to determine if any sharp changes in 
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contractor policy or environment had occurred which would not have 
been identified when that factor was used as a linear regression 
variable . 

a. Lockheed 

(1 ) Analysis of Variance . Lockheed started and completed 
30 contracts during the period of the data. The Lockheed data were 
pooled into two-year treatments to provide adequate numbers of observa 
tions in each cell and the odd contract was dropped. The sample sizes 
means and standard deviations which resulted were: 



Treatment 


Sample 

Size 


Mean 


Standard 

Deviation 


1 9k9 & 1930 


3 


-0.73 


3.92 


1931 Sc 1932 


10 


-It. 99 


3.83 


1 933 & 1 931i 


3 


-3.37 


U.30 


1 933 & 1 936 


k 


0.68 


2.1x2 


1 937 & 1 938 


3 


It. 67 


3.23 


1939 & 1 960 


6 


-3.07 


308 



The F-ratio for this analysis was k.5k which allows 
rejection of the one -population hypothesis at a significance level oi 

.003. Because the hypothesis was rejected at a very significant 

1 

level, the Scheffe comparison was performed. 

. . ’ 1 

(2) Scheffe Comparison . A Scheffe comparison was con- 
ducted using the F-value corresponding to a significance level of .03, 
"statistically significant". The pooled years 1 931 and 1 932 were de- 
termined to be significantly different from the pooled years 1937 and 
1938. However, in view of the sine-wave pattern exhibited by the Lock 
heed data when plotted against the year the contract was signed 
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(paragraph IV. 2. b), this appears to be a comparison of the peak of 
the wave with the valley, 
b. Grumman 

0 ) Analysis of Variance . Grumman started and completed 
2b contracts during the period of the data. These data were pooled 
into three-year treatments and a one-way analysis of variance was 
conducted. The samples sizes, means and standard deviations which 
resulted were: 



Treatment 


Sample 

Size 


Mean 


Standard 

Deviation 


1 9b9 - 1991 


6 


-10.33 


7.52 


1952 - 1 95b 


8 


-1 .59 


8.1*3 


1955 - 1957 


b 


-0.70 


10.22 


1961 - 1963 


b 


10.65 


10.65 



The F-ratio for this analysis was £.07 which allows 
rejection of the one -population hypothesis at the .025 level. 

(2) Scheffe Comparison . The Scheffe comparison was con- 
ducted using an F-value corresponding to a significance level of .02£. 
This level falls between "statistically significant" and "statistically 
very significant". Pooled years 1 9k9 - 1951 were found to be signifi- 
cantly different from pooled years 1961 - 1 963 * This finding agrees 
with the significant positive trend identified in the Grumman data in 
paragraph IV. 2. a. 

c . Douglas 

Douglas started and completed 2b contracts during the period 
under investigation. These data were pooled in two-year treatments 
and a one-way analysis of variance was performed. The sample sizes, 
means, and standard deviations from this analysis were: 



Treatment 


Sample 

Size 


Mean 


Standard 

Deviation 


1 91+9 & 1 950 


3 


-1 .17 


2.1+1+ 


1 951 & 1 952 


8 


-3.11+ 


12.23 


1 953 & 1 95 U 


6 


-3.20 


10.00 


1 955 & 1 956 


2 


3.1+5 


5-59 


1957 Sc 1958 


2 


-8.00 


9.05 


1959 Sc I960 


2 


-1 .80 


2.26 



The F-ratio for this analysis was 0.29 which is not statis 

i 

tically significant. No Scheffe comparison was conducted in this case 



V. CONCLUSIONS 



The most that can be gained from the results of the analyses pre- 
sented in Section IV alone are statistical inferences. Firstly, no 
cause/effect relationships can be established by statistical analysis. 
Secondly, failure to reject a one -population hypothesis or a K = 0 
hypothesis does not prove that those hypotheses were true. It merely 
states that with the statistical methods and standards employed and 
the data analyzed, it was not possible to reject those hypotheses. 
Nevertheless, the author will draw conclusions on the basis that the 
statistical analysis combined with the author's intuition and knowledge, 
and other sources of information present a reasonable case, lacking 
opposing evidence, for the conclusion drawn. The reader who is unwill- 
ing to accept the author's conclusions on such a basis is free to draw 
his own or none from the statistical evidence presented in Section IV 
of this paper. 

A. CONTRACT PARAMETERS' EFFECT ON DEVIATION FROM TARGET COST 

1 . Parameters Which Have a Significant Effect on Contract Cost 
Outcome 

a. All contractors 

(1 ) Length of the period of contract performance - Devia- 
tion from target cost varied positively with the length of performance 
of the contract at the rate of approximately 1 .$% per year. The 
author's hypothesis about this variable was supported by the data. 

In view of the length of time available for changes in both requirements 
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and details in a multi-year contract , and the general inflationary 
trend in this country since World War II, this is not a surprising 
conclusion. 

(2) Target profit - Deviation from target profit varied 
negatively as target profit at a rate of more than 1 $ per percent 
target profit increase. The conclusion that may be drawn here is: 
depending upon the sharing ratio, it may be less expensive in terms 
of total price for the government to be relatively generous in target 
profit. For example, in a contract with a 10$ target profit and a 
sharing ratio, a one percent increase in target profit may be expected 
to result in a 1 . 3 $ decrease in realized cost and a 0.3$ decrease in 
actual price. As the variations in target profit in the data were 
small, one would not be justified in making large target profit adjust- 
ments on this basis. The author's hypothesis with respect to target 
profit was not supported by the statistical analysis . The author has 
no intuitive explanation to offer for this rather surprising result. 

( 3 ) Ceiling price - Deviation from target cost varied 
positively with ceiling price at the rate of approximately O.U$ per 
percent increase in ceiling price. The author's hypothesis with re- 
spect to the effect of ceiling price was supported by the analysis. 

(U) The number of contracts signed in the Wavy aircraft 
and missile field during the same year as a particular contract - 
Deviation from target cost varied negatively as the number of contracts 
signed. This variable was chosen as a measure of the degree of compe- 
tition (or pressure) felt by a contractor when negotiating a particular 
contract. The theory being that as the total number of contracts in 
the Navy aircraft and missile field becomes smaller, the importance in 



terms of prime contractor survival of obtaining a particular contract 
goes up. Reaction to this pressure in the form of acceptance of an 
unrealistically low target cost does not require that the contractor 
knowingly buy in. All he has to do is to relax a little his normal 
degree of resistance to engineering optimism, etc. The author's hypo- 
thesis regarding the effect on deviation from target cost of reduced 
number of available contracts is supported by the analysis. One is 
reminded of the apocryphal story of the gambler who took part in a 
game he knew to be rigged against him because "It was the only game in 
town." The story may be apocryphal but its message is valid. 

( 5 ) The year the contract was signed - The deviation from 
target cost varied positively as the year the contract was signed. 

The author's hypothesis in this regard was supported by the analysis. 

The only explanation that occurs to the author for this result of the 
analysis is a gradual change with time of contractor policy toward cost 
overruns. It is significant that although the year the contract was 
signed and the number of contracts that year are rather closely related, 
they both ’are significant in the regression equation. Thus, one may 
not say that the positive effect of the year the contract was signed 
was simply a result of the intensified competitive atmosphere. 

(6) The product of the target cost and the number of 
items procured - Deviation from target cost varied negatively as the 
product of target cost and the number of items procured. The author's 
statistical reasoning concerning a combination of two unrelated vari- 
ables which singly were insignificant was supported by the appearance 
in the regression equation of this variable. The author was then faced 
with the necessity of attempting an explanation of this statistical 



fact in logical terms. This, he has been unable to do. Nevertheless , 
the predictive power of this variable remains whether or not the author 
can state a logical explanation. 

b . Individual Contractors 

(1 ) Grumman . Deviation from target cost in the case of 
Grumman contracts showed a strong reaction to both the year the contract 
was signed and the number of like contracts signed that year. The year 
the contract was signed alone accounted for 1;0$ of the variation in 
Grumman 1 s deviation from target cost. When all of the original variables 
and the two transgenerated ones were considered in a stepwise linear 
regression on the Grumman data, deviation from target cost varied nega- 
tively with the number of contracts signed and positively with the unit 
cost of the item procured (target cost/number of items procured). To- 
gether, these two variables accounted for 57$ of the variation in 
Grumman' s deviation from target cost. These two variables have approxi- 
mately twice the predictive power in Grumman 's case than the variables 
included in the regression equation for all contractors. No conclusions 
were drawn from the second regression (routine forced to use per 
IV. A. 2. a) because one of the variables included by the stepwise linear 
regression routine had only a .75 probability that its regression 
coefficient (b-j ) was not equal to zero. 

Grumman' s heavy dependence on Navy airframe contracts 
goes a long way toward explaining this result. The complexity involved 
in high unit cost items also provides a reasonable explanation of the 
positive variation of deviation from target cost with unit cost. The 
implications of this predictive equation with respect to the current 
cost overrun problems associated with the F-1 b program are obvious if 



one allows a valid comparison between non-total package procurement 
contracts and total package procurement ones. 

(2) Lockheed . The impression that Lockheed's deviations 
from target cost were cyclical with time was supported by the linear 
regression which used a cosine function of X^ as the independent 
variable. As compared to a linear regression using X^ as the indepen- 
dent variable, the correlation coefficient, determination coefficient 
and F- ratio were materially improved. The extent that the cosine 
equation accounts for the variation, 1 £$, Lockheed deviation from 
target cost oscillated about a level of -2.3$ with an amplitude of 
2.1$. Since the cyclical pattern does not occur when deviation from 
target cost is plotted against the number of contracts signed that 
year, one is tempted to postulate a cyclically varying management 
policy toward cost estimation or negotiated target cost. This theory 
would involve a more optimistic view of probable costs that is tempered 
as the results of the optimism begin to be apparent and then a shift 
back toward optimism as the results of relative pessimism begin to 
surface . 

(3) Lockheed and Grumman . Although the analysis of vari- 
ance conducted using the contractors as treatment levels provided no 
statistical basis for significant difference among contractors, the 
Lockheed and Grumman patterns of deviation from target cost were marked- 
ly different from the other contractors when plotted against the year 
the contract was signed. In both, trends over time were apparent. 

These time trends should be considered when negotiating contracts with 
two contractors. That is, Grumman 's tendency to underestimate or accept 
unrealistically low target costs had steadily increased while Lockheed 



oscillated about a rather favorable level of deviation from target 
cost (-2.3JG). 

2 . Statistically Insignificant Parameters 

Parameters which are unexpectedly insignificant can be as 
informative as those that prove to be significant. Accordingly , they 
will be given as much attention as were the significant parameters. 

a. Sharing ratio - The theory of cost -incentive contracting 
rests upon the supposed influence a share of underrun or overrun will 
have upon a contractor's cost-associated decisions. Further, multiple 
incentive contracts depend upon an assumption that the various per- 
formance and schedule incentives will be balanced against the cost 
incentive in the contractor's tradeoff decisions to the advantage of 
the contractor and the government. If the cost incentive proves to be 
ineffective and the results relatively unpredictable, then the supposed 
benefits of multiple incentive contracting must also be seriously 
doubted. This is a point of considerable importance since the addi- 
tional cost of administering a multiple -incentive contract may be 
largely wasted. 

In the case of the 177 Navy aircraft and missile contract 
examined in this thesis, sharing ratio is an insignificant parameter 
with respect to association with cost overrun/underrun (probability 
the b^ / 0 between 0.50 and 0.75). This is not a wholly surprising 
outcome in that a Rand study of 1|27 Air Force contracts resulted in a 
conclusion that the value of the sharing ratio was an insignificant 
parameter /~ ref. 57 • Because the Air Force contract data contained 
both cost incentive and cost reimbursable contracts, Rand was also 
able to conclude that although the value of the sharing ratio was 
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insignificant , deviations from target cost were significantly more 
positive in cost-reimbursable contracts than in cost-incentive con- 
tracts. An explanation which might be made is that the existence of 
the incentive completely unrelated to the value of the sharing ratio 
might influence contractor performance of the contract. A more likely 
explanation is that the use of an incentive contract in a particular 
case may have resulted in an upward shift in the negotiated target 
cost / ref. 67 • Since cost-reimbursable contracts encourage contrac- 
tors to grossly understate expected costs to ensure receiving the 
contract, cost-incentive contracts may have the benefit to the govern- 
ment of preventing grossly understated expected costs. 

In any case, the analysis contained herein in no way supports 
the theory that sharing the cost risk with the contractor significantly 
effects his efficiency of performance of the contract or influences him 
in cost-associated decision making. 

b. Target cost/Unit target cost - Neither target cost nor unit 
target cost were significantly associated with deviation from target 
cost. Some attractively logical hypotheses regarding these parameters, 
e.g., that lower unit. cost should be associated with more negative 
deviation from target cost because of economies of scale and the learn- 
ing curve. That is, one would suppose that the expected positive de- 
viation from target cost would be higher in a $100 M contract for 100 
items (unit cost of $1 M) than a $1 00 M contract for 10,000 items (unit 
cost of $10,000). That attractive theory must be allowed to wither for 
lack of empirical support. However, one might consider the possibility 
that contractors depend too heavily on economies of scale, etc., in 
their cost estimates, or that the discontinuities in production fre- 
quently resulting from the annual nature of defense funding may tend to 
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prevent gaining the cost advantages of economies of scale or the 
learning curve . 

c . Number of items - Similar arguments to those applied to 
unit cost could be applied to the number of items, i.e., the larger 
the number of items the more negative the deviation from target cost, 
but such arguments were similarly unsupported by the data. 

d. Contractor - No statistical significance could be attached 
to the contrasts among contractors. However, if faced with a choice 
between Kaman (mean percent deviation from target cost = 11 .95, standard 
deviation = 12.97) and Vertol (mean percent deviation from target cost = 
-ii . 77 j, standard deviation = 5.1|8) other things being essentially equal, 
one would be foolish to choose Kaman on the basis that the difference 

in contract experience with the two contractors was not statistically 
significant . 

e. Type of item - Although the items procured under the 177 
contracts in the data differ radically in physical terms, they were all 
procured by the same agency, NAVAIRSYSCOM, to aviation standards. No 
statistically significant differences were found among them. The means 
differ noticeably but the associated variances are too large to allow 
statistically significant differences among the means. 

B. OVERRUN/UNDERRUN PREDICTIVE MODEL (ALL DATA) 

The following discussion refers to the predictive model reported in 

paragraph IV. A. 1 .a, 

Percent deviation from target cost = 

-h2.h + 0.128X, - 1 .68X, + 0.397X k - 0.297X Q + 0.823X o - O.OOOOlOt . 

1 3 5 8 9 n 
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1 . Amount of Variation Explained 



The power of the model to explain the variation in the analyzed 
data, approximately 2$%, was disappointingly low. Its power to predict 
future contract outcomes, of course, is unknown but should not be ex- 
pected to improve over that for the data from which it was generated. 
Still, a model of this nature might give some inkling of the outcome 
of a particular future contract. It is probably better than an assump- 
tion that the contract outcome in terms of actual cost will be equal to 
the negotiated target cost. The argument here, albeit a weak one, is 
that the model is probably better than no consideration of the histori- 
cal realities of contract outcome. But, of course, past realities may 
or may not be similar to future realities. As is the case with many 
mathematical models, most of the value of this model is in understand- 
ing the relationships or lack of them among the variables. 

2. Standard Error of Estimate 

The large standard error of estimate, approximately Q%, is in 
keeping with the low amount of variation accounted for. The model as 
a valid predictor of future contract outcome is largely useless. 

C. THE RELATIONSHIP OF TARGET PROFIT TO OTHER CONTRACT PARAMETERS 
Three other contract parameters were found to be statistically 
related to negotiated target profit. They were sharing ratio, ceiling 
price, and number of items procured. Together these variables follow 
the previous trend, that they account for little of the variation of 
target profit. This discussion will concern the individual variables. 

1 . Sharing ratio - Target profit varied positively with sharing 
ratio. The theory that the contractor who bears the larger financial 
risk should be rewarded with a higher target profit appears to be 
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supported in this case. A similar conclusion was reached by Rand in 
its study of k27 Air Force contracts /""ref. 7 _7 • 

2. Ceiling price - The negotiated target profit varied negatively 
with ceiling price. A higher ceiling price reduces the contractor’s 
financial risk by causing the point of total assumption (PTA) to occur 
further from the target cost. All other things being equal, the further 
the PTA is from the target cost, the lower the probability of the con- 
tractor finding himself in the position of paying excess costs out of 
his own pocket, dollar for dollar. Again, the lower the financial risk, 
the lower target profit, and vice versa. 

3. Number of items procured - The target profit varied positively 
as the number of items procured. No explanation for this phenomena was 
apparent to the author. 

D. SUMMARY OF CONCLUSIONS 

The statistical inference really worthy of note in this thesis is 
the repeated failure of empirical evidence to support the supposed 
effectiveness of the sharing ratio. Since the cost incentive is the 
cornerstone of incentive contracting and incentive contracting is in 
extensive use today, it is a little disconcerting to find no empiricaj. 
evidence to support it, i.e., percent deviation from target cost is 
not statistically related to the sharing ratio. It would seem prudent, 
lacking supportive evidence, to not place such dependence on incentive 
contracting to influence efficient contractor performance . Faith in 
an improvable principle has its place in religious matters but not, in 
the author's view, in matters of economy and national defense. 
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APPENDIX A 



COMPUTER CODE FOR THE SCHEFFE 
COMPARISON (S-TEST) 



The following computer code is written in FORTRAN G and was used on 
an IBM 360 computer. 

C PROGRAM TO TEST FOR SIGNIFICANT DIFFERENT IN ANOVA TREATMENTS 

C USING SCHEFFE METHOD 

READ (5,10) NGROUP, SQUARE, FVAL 
10 FORMAT (12, 2FlO.it) 

DIMENSION ISAMSZ ( 30 ) , ERMEAN ( 30 ) , ERTOT( 30 ) 

DO 1 3 J=1 ,NGROUP 

READ (3,12) ISAMZ (J), ERMEAN (J ) 

12 FORMAT (12, FI 2.3) 

ERTOT (J )= ISAMSZ ( J )*ERMEAN ( J ) 

13 CONTINUE 
JGR0UP=NGR0UP-1 

A- ( JGR0UP-::-F7AL)-x-x-0. 3 
DO 200 1=1 , JGROUP 
DO 130 J=I, JGROUP 
K=J+1 

COMP=ISAMSZ (K )-x-ERTOT( I ) -ISAMSZ (I )-::-ERTOT(K ) 

IF (COMP.GE.O) GO TO 132 
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COMP=-COMP 



1 52 V ARCOM=I SAMSZ (K)**2*ISAMSZ (i )-x-SQ T JARE+ISAMSZ (I 2 *ISAMSZ(K )-x-SQUARE 
AVAR= A-x-V ARCOM-x-x-O . 5 
WRITE (6,110) I, K, COMP, AVAR 

110 FORMAT ( '0' , 'GROUP' ,2X,I2,itX, 'GROUP' ,2X,I2,ljX, 'COMPARISON* 1 ' , 

1 FI 2 . k, IjX , ' AVARIANCE= 1 , FI 2 . I 4 ) 

IF ( COMP. LE .AVAR) GO TO 1 50 
WRITE (6,120) I,K 

120 FORMAT ( ' 0 ' , ' -x-x-x-x- GROUP' , IX, 12, IX, 'IS SIGNIFICANTLY DIFFERENT FROM 
1 GROUP' IX, I 2, ' **#*') 

150 CONTINUE 
200 CONTINUE 
STOP 
END 



NGROUP 

SQUARE 



FVAL 



Explanation of Input Variables 

Number of treatment groups (levels) 
Within-groups mean square value from 
preceeding ANOVA 

F ratio for desired level of significance 
of the comparison based on be tween-groups 
and within-groups degrees of freedom from 
the ANOVA. Example: If the between groups 

degrees of freedom are 3 and the within- 
groups 18, and the desired level of test is 
.025 3 the value of FVAL is 3-9539- 
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(F 3,18 ; .025” 3 - 9539) - 

ISAMSZ(j) The sample size in the jth treatment group 

ERMEAN(J) The mean of the jth treatment group 

Limitati ons 

The program as contained herein is limited to: (1 ) no more than 

30 treatment groups and accompanying values of ISAMSZ and ERMEAN, and 
(2) comparison of each treatment group with each of the other treat- 
ment groups . 



Sample Output 



GROUP 1 GROUP 1* COMPARISON 1*93.3989 AVARIANCE* 1*36.9731 
**** GROUP 1 IS SIGNIFICANTLY DIFFERENT FROM GROUP 1* *##* 

GROUP 2 GROUP 3 






LIST OF REFERENCES 



1 . Scherer, F.M., "The Theory of Contractual Incentives for Cost 

Reduction," The Quarterly Journal of Economics, v. 78, p. 267, 
May 196U. 

2. Scherer, F.M., "The Theory of Contractual Incentives for Cost 

Reduction," The Quarterly Journal of Economics, v. 78, p. 262, 
May 1 96k • 

3 • Rand Corporation Report RM-51 20-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 1 I 4 , 
September 1966.' 

J 4 . McCall, J.J., "The Simple Economics of Incentive Contracting," 

The American Economic Review, v. 60, number 5, p. 8/i5, 
December 1970. 

3* Rand Corporation Report RM-51 20-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 38, 
September 1 966 . 

6. Rand Corporation Report RM-51 20-PR, Cost Incentives and Contract 

Outcomes : an Empirical Analysis , by Irving N. Fisher, p. If 3, 
September 1 906 / 

7. Rand Corporation Report RM-51 20-PR, Cost Incentives and Contract 

Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 2b, 
September 19^6. 



\ 



55 



INITIAL DISTRIBUTION LIST 



No. 



1 . Defense Documentation Center 
Cameron Station 
Alexandria, Virginia 22311; 

2. Library, Code 0212 
Naval Postgraduate School 
Monterey, California 939l;0 

3. Associate Professor Michael G. Sovereign 
Code 55Zo 

Department of OR/AS 
Naval Postgraduate School 
Monterey, California 939kO 

1;. Professor P.A.W. Lewis 
Code 55Lw 

Department of OR/AS 
Naval Postgraduate School 
Monterey, California 939^0 

5. Mr. M.E. Biciocchi 
Code MAT 022B 

Headquarters, Naval Material Command 
Washington, D.C. 20360 

6. Assistant Professor D.R. Whipple 
Code 55Wp 

Department of OR/AS 
Naval Postgraduate School 
Monterey, California 939i;0 

7. Department of OR/AS 
Naval Postgraduate School 
Monterey, California 939k 0 



Copies 

2 

2 

1 

1 

1 

1 



56 



Se c u ri ty Classification 



DOCUMENT CONTROL DATA - R L D 

(SerurifK c las si ( tc mtion of title, body of abstract and indexing annotation must be entered when the overall report Is classified) 



r.inaTING activity (Corporate author) 

L-ral Postgraduate School 
'iiterey, California 939UO 



2a. REPORT SECURITY CLASSIFICATION 



Unclassified 



2b. GROUP 



(ORT TITLE 



.Statistical Analysis of Deviations from Target Cost in NAVAIRSYSCOMHQ Fixed-Price 
Leentive Contracts During the 1 9h9 - 1 963 Time Frame 

D.CRIPTIVE notes (Type of report and, Inc I us ive dates) 

Ister's Thesis; March 1 °73 

rrHOR(S) (First name, middle Initial, last name) 

Yx Wayne Dixon 



J >OR T DATE 



vrch 1 973 



7 a. TOTAL NO. OF PAGES 

38 



7b. NO. OF REFS 

7 



JNTRACT OR GRANT NO. 



*OJEC T NO- 



9 a, ORIGINATOR’S REPORT NUMBER(S) 



9b. OTHER REPORT NOls) (Any other numbers that may be a a signed 
this report) 



ISTRI8UTION STATEMENT 



Approved for public release; distribution unlimited. 



IPPLEMENTARY NOTES 



12. SPONSORING MILITARY ACTIVITY 

Naval Postgraduate School 
Monterey, California 939UO 



bstract 



This paper statistically analyzes 13 years of NAVAIRSYSCOMHQ fixed-price incen- 
ive contract experience in the aircraft and missile procurement field. The relation 
f basic contract parameters to contract outcome is explored through regression and 
lalysis of variance techniques. 

The inferences arising from the statistical analysis are combined with other 
-[formation to draw conclusions regarding incentive contracting. The most important 
f these is that there is no evidence that the negotiated sharing ratio has any 
ifluence on the contractor during the performance of the contract. 



FORM <^70 (PAGE 1 ) 
/ t no v e 5 I *v / V 

0101 - 807 - 681 1 



37 



Security Cl Attifi cation 



A- 3 l 40 a 



Security Cla§ s if iration 



KEY WORD* 


LINK A 


LINK B 


LINK C 




HOLE 


W T 


ROLE 


W T 


ROLE 


W T 


Statistical Analysis 
Cost-Incentive Contracting 
Fixed-Price Incentive 
Sharing Ratio 

Aircraft and Missile Contracts 

Naval Air Systems Command 

Ceiling Price 

Target Profit 

Grumman 

Lockheed 















D 



,"'“..1473 * BACK ) 



010! *807-6821 



58 



Security Classification 



a - 3 1 409 



'"hes i s 1 8 may 79 2 

0599 Dixon , - t 8 

c.l A statistical analysis 

of deviations from targety S i s 
cost in NAVAIRSYSCOMHQ arget 
fixed-price incentive hQ 

contracts during the e 

1949-1965 time frame. 




Thes i s 

D599 

c.l 



Dixon 



.l4o308 



A statistical analysis 
of deviations from target 
cost in NAVAIRSYSCOMHQ 
fixed-price incentive 
contracts during the 
1949-1965 time frame. 



thesD599 

A statistical analysis of deviations fro 




mi n urn < in. ilium 111 iiiiii i 
3 2768 002 24753 8 

DUDLEY KNOX LIBRARY 



